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Abstract 

Dr. Ougrin's evaluation of cognitive therapy versus exposure in anxiety disorders reported a standardised mean 
difference [SMD] (95% confidence interval [CI]) of 0.52 (0.37, 0.74) for short-term outcomes and 0.46 (0.29, 0.73) for 
long-term outcomes in social phobia, and 0.88 (0.69, 1.1 1) for short-term outcomes and 1.05 (0.80, 1.37) for 
long-term outcomes in posttraumatic stress disorder (PTSD). These were incorrectly meta-analysed. Upon 
re-analysis, we found that the correct SMD (95% CI) was -0.66 (-1.19, -0.14) for short-term outcomes and mean 
difference (95% CI) of -29.66 (-46.13, -13.19) on the Social Phobia subscale from the Social Phobia Anxiety 
Inventory for long-term outcomes in Social Phobia. For PTSD, the SMD (95% CI) for short-term outcomes was -0.13 
(-0.36, 0.1 1) and 0.05 (-0.22, 0.32) for long-term outcomes. However, correcting the errors did not change the 
interpretation of the findings considerably. 
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We read with great interest the systematic review evaluat- 
ing the efficacy of exposure versus cognitive therapy (CT) 
in anxiety disorders, performed by Dr. Dennis Ougrin [1]. 
The author meta-analysed 20 randomised controlled trials 
(RCTs) comparing CT versus exposure in four anxiety dis- 
orders and concluded the following: "there appears to be 
no evidence of differential efficacy between cognitive ther- 
apy and exposure in PD [panic disorder], PTSD [posttrau- 
matic stress disorder] and OCD [obsessive compulsive 
disorder] and strong evidence of superior efficacy of cog- 
nitive therapy in social phobia" [1]. 

Upon reviewing the short-term and long-term results 
of CT versus exposure in patients with Social Phobia, we 
found two errors. First, the summary effect (standard 
error [SE]) for the short-term outcome in Hofmann 
(2004) was incorrectly inputted as -0.28 (0.26). We con- 
tacted Dr. Stefan Hofmann [2], obtained the raw data 
from the published trial, and found that the correct 
summary effect (SE) was -0.19 (0.27). Second, Dr. 
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Ougrin states, "the overall effect (the end-of-treatment 
standardised mean difference (SMD), Hedge s g) is sum- 
marised in Figure 7" [1]. The forest plot, however, does 
not indicate what summary effect was reported in the 
pooled analysis and thus the assumption was made that 
it was the SMD. Upon replicating the meta-analysis, we 
found that the summary effect was incorrectly pooled as 
an odds ratio (OR), i.e. OR (95% confidence interval 
[CI]) of 0.52 (0.37, 0.74) for short-term outcomes. Thus, 
we inputted the correct summary effect for Hofmann 
2004 and re-analysed the meta-analysis using SMD for 
our summary effect, as originally intended. We found 
that the corrected SMD (95% CI) was -0.66 (-1.19, 
-0.14) for short-term outcomes. We used the random 
effects model given that the i 2 was 56% (Figure 1). Using 
the Cohens d criteria of 0.2 to represent a small effect, 
0.5 a medium effect and 0.8 a large effect [3], we found 
that CT has a medium effect in improving social phobia 
versus exposure with the lower bounds of the 95% CI in 
the range of a modest effect, for short-term outcomes. 

Long-term outcomes for social phobia were also incor- 
rectly pooled as an OR. We found that the corrected 
SMD (95% CI) was 0.46 (0.29, 0.73) for long-term 
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Std. Mean Difference 

Study or Subgroup Std. Mean Difference SE Weight IV, Random, 95% CI 



Std. Mean Difference 
IV, Random, 95% CI 



Clark 2003 
Clark 2006 
Hofmann 2004 



-0.87 0.33 31.4% -0.87 [-1.52. -0.22] 
-1.01 0.33 31.4% -1.01[-1.66,-0.36] 
-0.19 0.27 37.2% -0.19 [-0.72. 0.34] 




Total (95% CI) 100.0% -0.66 1-1.19, -0.14] 

Heterogeneity: Tau 2 = 0.12: Chi 2 = 4.51. df = 2 (P = 0.11): I 2 = 56% 

Test for overall effect: Z = 2.47 (P = 0.01) Favours Cognjtjve Therapy FavQurs Exposure 

Figure 1 Corrected meta-analysis of Figure 7 in Ougrin (2011): the short-term efficacy of cognitive therapy versus exposure in Social 
Phobia. 



outcomes. However, the SMD should only be used when 
pooling different assessments for the same outcome. If 
the same assessment is reported across trials, the mean 
difference (MD) is always preferred. Both Clark (2006) 
and Hofmann (2004) [2,4] reported social phobia sub- 
scale scores from the Social Phobia and Anxiety Inven- 
tory (SPAI) [5]. Thus, we calculated the MD (95% CI) as 
-29.66 (-46.13, -13.19) (Figure 2). 

Additional errors identified by Professor Julio 
Sanchez-Meca 

Professor Julio Sanchez-Meca, who was a reviewer of 
our article, identified additional errors in Dr. Ougrins 
paper as follows: 

1. In Figure 3 in Ougrin (2011), the forest plot was 
incorrectly pooled as odds ratios when the effect 
sizes were in fact SMDs (see Table 2 in Ougrin). We 
found that the corrected SMD (95% CI) was -0.13 
(-0.36, 0.11) for short-term outcomes in PTSD 
(Figure 3). 

2. Figure 4 in Ougrin (2011) presents the same error as 
above. We found that the corrected SMD (95% CI) 
was 0.05 (-0.22, 0.32) for long-term outcomes in 
PTSD (Figure 4). 

3. On page 4 in Ougrin (2011), second column, fifth 
paragraph, reporting the results of the meta-analysis 



for short-term outcomes in panic disorder, an I 2 of 
68% is reported. However, in Figure 5 in Ougrin 
(2011), an I 2 of 62% is reported. 
4. On page 4 in Ougrin (2011), second column, last 
paragraph, reporting the results of the meta-analysis 
for long-term outcomes in panic disorder, an I 2 of 24% 
is reported. However, in Figure 6 in Ougrin (2011), an 
I 2 of 69% is reported. In addition, the author reported 
that a fixed effects model was applied when actually, a 
random-effects model was employed. 

Conclusions 

In summary, after correcting and re-analysing the meta- 
analyses, we found a medium effect in CT improving so- 
cial phobia in comparison to exposure. It is important to 
note, however, that correcting the errors did not change 
the interpretation of the findings considerably in Dr. 
Ougrins paper. Our paper stresses the importance of 
critically checking ones data and analyses to ensure 
validity of results. 

Response 

By Dennis Ougrin dennis.ougrin@kcl.ac.uk 

I would like to thank Ebrahim and Bance for confirm- 
ing and essentially independently replicating the main 
results of the meta analysis evaluating the efficacy of 



Study or Subgroup 



Cognitive Therapy 
Mean SD Total 



Exposure 
Mean SD Total 



Mean Difference 
Weight IV, Fixed, 95% CI 



Mean Difference 
IV, Fixed, 95% CI 



Clark 2006 
Hofmann 2004 



52.96 
84.46 



41.68 
33.81 



21 
17 



L09 



39.16 
35.17 



Total (95% CI) 38 

Heterogeneity: Chi 2 = 0.45. df = 1 (P = 0.50): I 2 = 0% 
Test for overall effect: Z ■ 3.53 (P ■ 0.0004) 



21 45.3% -35.84[-60.30.-11.38] 
20 54.7% -24.54 [-46.81. -2.27] 

41 100.0% -29.66[-46.13,-13.19] 




Favours Cognitive Therapy Favours Exposure 



Figure 2 Corrected meta-analysis of Figure 8 in Ougrin (2011): the long-term efficacy of cognitive therapy versus exposure in Social 
Phobia. 
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Study or Subgroup Std. Mean Difference SE Weight 



Std. Mean Difference 
IV, Fixed, 95% CI 



Std. Mean Difference 
IV, Fixed, 95% CI 



Foa1991 
Foa1999 
Marks 1 998 
Resick2002 
Tarrier 1999 



-0.48 
0.14 
0.28 

-0.39 
0.09 



0.37 
0.31 
0.33 
0.19 
0.26 



10.5% 
15.0% 
13.2% 
39.9% 
21.3% 



Total (95% CI) 100.0% 
Heterogeneity: Chi 2 = 5.79, df = 4 (P = 0.22); I 2 = 31 % 
Test for overall effect: Z = 1 .07 (P = 0.28) 



-0.48 [-1.21, 0.25] 
0.1 4 [-0.47, 0.75] 
0.28 [-0.37, 0.93] 

-0.39 [-0.76, -0.02] 
0.09 [-0.42, 0.60] 



-0.13 [-0.36, 0.11] 




-2-1 0 1 

Favours Cognitive Therapy Favours Exposure 



Figure 3 Corrected meta-analysis of Figure 3 in Ougrin (2011) "Cognitive therapy versus exposure for PTSD. Meta-analysis: short-term 
outcomes". 



exposure versus cognitive therapy (CT) in anxiety disor- 
ders [5], Ebrahim and Bance reviewed the short-term 
and long-term results of CT versus exposure in patients 
with social phobia and found a slightly different standar- 
dised mean difference (SMD) and standard error [SE] 
for the short-term outcomes in Hofmann (2004) [1]: 
-0.19 (0.27) rather then -0.28 (0.26). Ebrahim and Bance 
did not state how their calculation was made. Since dif- 
ferent formulae exist for this calculation it might be 
helpful if the authors provide the entirety of their calcu- 
lations rather then only providing the final figure. The 
authors further used their calculation of the SMD in 
Hofmann (2004) [1] to re-calculate the pooled SMD, 
using random effects model instead of the original fixed 
effects model (due to the increased heterogeneity) and 
found the pooled SMD to be -0.66 (-1.19, -0.14), 
slightly greater then the original estimate but with wider 
confidence intervals. The recalculation of long-term out- 
comes for social phobia yielded essentially similar 
results. However, the authors substituted the SMD with 
the mean difference (MD) from the Social Phobia and 
Anxiety Inventory (SPAI) as the SPAI results were 
reported by all three studies used to calculate the pooled 



SMD [1-6]. This may be a preferred strategy in most 
cases, however for the purpose of the original meta ana- 
lysis and a priori decision was made to use the main out- 
come measure in each study. In both Clark (2003) and 
Clark (2006) the main outcome measure was a social 
anxiety composite, hence the decision was to use the 
SMD instead of the MD. Ebrahim and Bances recalcu- 
lation of the SMD in the PTSD [posttraumatic stress dis- 
order] studies yielded essentially similar results as those 
in the original meta analysis. I would like to thank the 
authors for spotting several typos and some reporting in- 
consistencies which sadly even a very thorough peer re- 
view may miss. One important lesson is to check all 
default settings in the software used for meta analyses. Fi- 
nally I would like to emphasise that confirmations and in- 
dependent replications are extremely important and still 
quite rare in the field of psychological therapies. Both clini- 
cians and researchers may feel more confident that extant 
literature does indeed indicate no overall difference be- 
tween cognitive therapy (including behavioural experi- 
ments) and exposure for in PD [panic disorder], PTSD and 
OCD [obsessive compulsive disorder] and points to super- 
ior efficacy of cognitive therapy in social phobia. 



Study or Subgroup Std. Mean Difference SE 



Std. Mean Difference 
Weight IV, Fixed, 95% CI 



Std. Mean Difference 
IV, Fixed, 95% CI 



Foa1991 
Foa1999 
Marks 1998 
Resick2002 
Tarrier 1999 



0.21 0.36 14.6% 

0.16 0.31 19.7% 

0.83 0.43 10.3% 

-0.18 0.19 52.5% 

-0.26 0.81 2.9% 



Total (95% CI) 

Heterogeneity: Chi 2 = 5.22, df= 4 (P= 0.27); l 2 = 23% 
Test for overall effect: Z=0.33(P = 0.74) 



100.0% 



0.21 [-0.50,0.92] 
0.1 6 [-0.45, 0.77] 
0.83 [-0.01, 1.67] 
-0.1 8 [-0.55, 0.1 9] 
-0.26 [-1.85,1.33] 

0.05 [-0.22, 0.32] 



-h 



-2-1 0 1 

Favours Cognitive Therapy Favours Exposure 

Figure 4 Corrected meta-analysis of Figure 4 in Ougrin (2011) "Cognitive therapy versus exposure for PTSD. Meta-analysis: long-term 
outcomes". 
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